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DETAILED ACTION 

Response to Arguments 

1 . Applicant's arguments with respect to claims 1 -6, 9, 1 0, 1 2-1 4, 27, 28, and 
30-40 have been considered but are moot in view of the new ground (s) of 
rejection. 

Claim Rejections - 35 USC § 103 

2. The text of those sections of Title 35, U.S. Code not included in this action 
can be found in a prior Office action. 

3. Claims 1-6, 9, 10, 12-24, 27, 28, and 30-40 are rejected under 35 
U.S.C. 103(a) as being unpatentable over Drews et al. (previously cited and 
hereinafter Drews), US 5,768,607 A, in view of Baumgartner et al. (previously cited 
and hereinafter Baumgartner), US 5,642,171 A, and Heinz et al. (hereinafter 
Heinz), US 5,81 6,820 A. 

4. Regarding claim 1, Drews teaches a method for synchronizing operations 
in a computer environment with accompanying audio (see column 3, line 15 - 
column 4, line 2), said method comprising: 

replaying sa/'d operations and said accompanying audio in said computer environment, 
said operations resulting from processing of recorded user inputs (see column 5, lines 37-57, 
and column 7, line 12 - column 8, line 52); 

creating a synchronization point at a common point in said replaying of said operations 
and said accompanying audio (see column 13, lines 30-36); 

associating said synchronization point with said accompanying audio, said 
synchronization point providing a reference point to substantially synchronize said 
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accompanying audio when said operations are replayed in a replay computer environment 
using said recorded user inputs (see column 13, lines 30-36); 

detecting said synchronization point during a subsequent replay of said operations and 
said accompanying audio in said replay computer environment, said subsequent replay 
involving another processing of said recorded user inputs (see column 14, lines 4-26); 

comparing said synchronization point with a time value associated with said another 
processing of said recorded user inputs (see column 14, lines 4-48) ; 

selectively pausing said subsequent replay of said accompanying audio if a difference 
between said synchronization point and said time value exceeds a predefined amount so that 
said subsequent replay of said operations can catch up to said accompanying audio; and 

resuming said subsequent replay of said accompanying audio if a difference between 
said synchronization point and a current time value does not exceed a second predefined 
amount, said current time value being associated with said another processing of said recorded 
user inputs. 

Drews teaches a method, wherein operations with accompanying audio is 
synchronized in a computer environment. However, Drews does not teach 
pausing accompanying audio if the operations (i.e. recorded user inputs) fall 
behind the synchronization, so that they video can catch up to the audio 
portion of the recording. 

Baumgartner teaches a method of synchronizing audio and video 
streams in a computer environment (see abstract). Specifically, Baumgartner 
teaches an environment, wherein the processing capabilities of the system are 
too slow (see column 5, lines 18-27). It is not hard for one of ordinary skill in the 
art at the time of the invention to appreciate that a video processor may have 
the same deficiencies. Baumgartner teaches pausing the audio if the audio is 
too far ahead of the video playback (see column 7, lines 1 6-23). Specifically, 
Baumgartner teaches the selectively pausing and resuming steps (see figure 5A, 



steps 518, 520, 522, and 524 and column 14, lines 1-26). It would have been 
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obvious for one of ordinary skill in the art at the time of the invention to combine 
the teachings of Drews and Baumgartner for the purpose of synchronizing audio 
and video (i.e. recorded user input) on slower computer hardware. However, 
the combination does not appear to teach replaying the operations and 
accompanying audio using an event recording, wherein the event recording 
includes all user inputs during event recording and specifically, initial conditions 
of the original computing environment when the event recording was initiated. 

Heinz teaches a recording of a visual display in a computing environment. 
Specifically, Heinz teaches recording all user inputs are during an event 
recording, or session (see column 4, lines 48-58). Furthermore, Heinz teaches that 
initial conditions are recorded so that a training sequence will work as originally 
intended (see column 2, lines 5-21 ). It would have been obvious for one of 
ordinary skill in the art at the time of the invention to combine the teachings of 
Drews, Baumgartner, and Heinz for the purpose of providing consistent results in 
a session recording playback. 

5. Regarding claim 2, the further limitation of claim 1 , see the preceding 
argument with respect to claim 1 . The combination teaches the method of 
claim 1, wherein 

creating of said synchronization point includes creating said synchronization point in 
response to a user command (see Drews, column 13, lines 30-36, wherein a user 
recording sound and cursor movement causes the creation of synchronization 
points). 
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6. Regarding claim 3, the further limitation of claim 1 , see the preceding 
argument with respect to claim 1 . The combination teaches the method of 
claim 1, wherein 

said common point is at a point in time where there is no audio output during said 
replaying of said accompanying audio (see Drews, column 13, lines 37-45 and figure 
1 la, wherein common points are created at the beginning of recording, when 
there is no sound). 

7. Regarding claim 4, the further limitation of claim 1 , see the preceding 

argument with respect to claim 1 . The combination teaches the method of 

claim 1, further comprising 

obtaining a current time value associated with said processing of said recorded user 
inputs, said current time value corresponding to said synchronization point (see 
Baumgartner, figures 5a and 5b and column 13, lines 23-67, wherein the current 
time value is implied to be a current audio or video time index). 

8. Regarding claim 5, the further limitation of claim 1 , see the preceding 
argument with respect to claim 1 . In the combination, Baumgartner teaches 
different ways of storing the video and the audio (see column 1 0, lines 30-39). 
Baumgartner does not appear to teach a method of claim 1 , wherein the 
synchronization point is saved in the first file with the recorded audio, but it 
would have been obvious for one of ordinary skill in the art at the time of the 
invention to place the synchronization points in either file, or a single file with 
interleaved data. It would have been an obvious design choice. 

9. Regarding claim 6, the further limitation of claim 1 , see the preceding 
argument with respect to claim 1 . The combination teaches the features of 
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claim 1, and makes these features obvious. In the combination, Drews teaches 
the editing of the user commands, or drawings, and editing of the recorded 
sound (see column 1 0, lines 30-40). It is inferred by the editing, or appending, of 
the recording will change a time value of the synchronization points. Drews 
teaches a timeline (see figure 1 la, 1 lb, and 12). It would have been obvious for 
one of ordinary skill in the art at the time of the invention to allow direct editing 
of the timeline by moving a positional marker on the timeline as is known in 
video editing. 

1 0. Regarding claim 9, the further limitation of claim 1 , see the preceding 
argument with respect to claim 1 . The combination teaches the method of 
claim 1, wherein 

said second predefined amount equals said predefined amount (see Baumgartner, 
column 14, lines 1-26 teaches one embodiment, wherein the predefined 
amounts are different). 

In the combination, Baumgartner teaches a preferred embodiment, wherein it is 
obvious to change the predetermined amounts according to different 
applications. It would have been obvious for one of ordinary skill in the art at 
the time of the invention to use the same predefined amounts or thresholds to 
pause or resume the audio in order to make the comparison of the current time 
value with the synchronized playback value easier. 
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1 1 . Regarding claim 10, see the preceding argument with respect to claim 1 . 
The combination teaches the method of claim 1 , which reads on the method of 
claim 10. 

12. Regarding claim 12, the further limitation of claim 10, see the preceding 
argument with respect to claim 9. The combination makes obvious the method 
of claim 10 with these additional features. 

1 3. Regarding claim 13, the further limitation of claim 1 0, see the preceding 
argument with respect to claim 6. The combination makes obvious the method 
of claim 10 with these additional features. 

1 4. Regarding claim 14, the further limitation of claim 1 0, see the preceding 
argument with respect to claim 1 . The combination teaches the features of 
claim 10 and these additional features. 

15. Regarding claim 15, the further limitation of claim 14, see the preceding 
argument with respect to claim 2. The combination teaches the features of 
claim 14 and these additional features. 

1 6. Regarding claim 1 6, the further limitation of claim 1 4, see the preceding 
argument with respect to claim 3. The combination teaches the features of 
claim 14 and these additional features. 

1 7. Regarding claim 1 7, the further limitation of claim 1 4, see the preceding 
argument with respect to claim 5. The combination teaches the features of 
claim 14 and these additional features. 
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18. Regarding claim 18, the further limitation of claim 14, see the preceding 
argument with respect to claim 6. The combination makes obvious the method 
of claim 14 with these additional features. 

1 9. Regarding claim 1 9, see the preceding argument with respect to claim 1 . 
The combination teaches the method of claim 1, which reads on the storage 
medium readable by a computer, tangibly embodying a program of 
instructions executable by said computer to perform method steps for 
synchronizing operations in a computer environment with accompanying audio, 
said method comprising the features of claim 1 . 

20. Regarding claim 20, the further limitation of claim 1 9, see the preceding 
argument with respect to claim 2. The combination teaches the features of 
claim 2, which reads on the features of claim 20. 

21 . Regarding claim 21 , the further limitation of claim 1 9, see the preceding 
argument with respect to claim 3. The combination teaches the features of 
claim 3, which reads on the features of claim 21 . 

22. Regarding claim 22, the further limitation of claim 1 9, see the preceding 
argument with respect to claim 4. The combination teaches the features of 
claim 4, which reads on the features of claim 22. 

23. Regarding claim 23, the further limitation of claim 1 9, see the preceding 
argument with respect to claim 5. The combination teaches the features of 
claim 4, which reads on the features of claim 23. 
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24. Regarding claim 24, the further limitation of claim 1 9, see the preceding 
argument with respect to claim 6. The combination makes obvious the method 
of claim 19 with these additional features. 

25. Regarding claim 27, the further limitation of claim 19, see the preceding 
argument with respect to claim 9. The combination teaches the features of 
claim 4, which reads on the features of claim 27. 

26. Regarding claim 28, see the preceding argument with respect to claim 1 . 
The combination teaches the method of claim 1, which reads on the storage 
medium readable by a computer, tangibly embodying a program of 
instructions executable by said computer to perform method steps for 
synchronizing operations in a computer environment with accompanying audio, 
said method comprising the features of claim 10. 

27. Regarding claim 30, the further limitation of claim 28, see the preceding 
argument with respect to claim 9. The combination makes obvious the storage 
medium of claim 28 with these additional features. 

28. Regarding claim 31, the further limitation of claim 28, see the preceding 
argument with respect to claim 6. The combination makes obvious the method 
of claim 28 with these additional features. 

29. Regarding claim 32, the further limitation of claim 28, see the preceding 
argument with respect to claim 1 . The combination teaches the features of 
claim 28 and these additional features. 
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30. Regarding claim 33, the further limitation of claim 32, see the preceding 
argument with respect to claim 2. The combination teaches the features of 
claim 32 and these additional features. 

31 . Regarding claim 34, the further limitation of claim 32, see the preceding 
argument with respect to claim 3. The combination teaches the features of 
claim 32 and these additional features. 

32. Regarding claim 35, the further limitation of claim 32, see the preceding 
argument with respect to claim 5. The combination teaches the features of 
claim 32 and these additional features. 

33. Regarding claim 36, the further limitation of claim 32, see the preceding 
argument with respect to claim 6. The combination makes obvious the method 
of claim 32 with these additional features. 

34. Regarding claim 37, see the preceding argument with respect to claim 

1 6. The combination teaches a method with the combined features of claims 
10, 14, and 16. 

35. Regarding claim 38, see the preceding argument with respect to claim 
37. The combination teaches the method of claim 1 6, which reads on the 
storage medium readable by a computer, tangibly embodying a program of 
instructions executable by said computer to perform method steps for 
synchronizing operations in a computer environment with accompanying audio, 
said method comprising the features of claim 37. 
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36. Regarding claim 39, see the preceding argument with respect to claim 
1 7. The combination teaches a method with the combined features of claims 
10, 14, and 17. 

37. Regarding claim 40, see the preceding argument with respect to claim 
39. The combination teaches the method of claim 1 7, which reads on the 
storage medium readable by a computer, tangibly embodying a program of 
instructions executable by said computer to perform method steps for 
synchronizing operations in a computer environment with accompanying audio, 
said method comprising the features of claim 39. 

Conclusion 

38. The prior art made of record and not relied upon is considered pertinent 
to applicant's disclosure. 

Owen, US 5,585,583 A, (previously cited) teaches mixing a student's 
musical instrument with a synchronized audio/video playback of an instructional 
musical piece (abstract, column 3, line 57 - column 4, line 9, column 6, lines 43- 
59, column 7, lines 1 5-26, column 7, line 56 - column 8, line 1 6 and column 1 3, 
lines 48); 

Kuribayashi, US 5,615,189 A, (previously cited) teaches synchronization of a 
word clock in MIDI circuits (abstract and column 1 , line 50 - column 2, line 20); 
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Berstis et al., US 5,903,266 A, (previously cited) teaches audio instructions 
detailing various steps of connecting equipment (column 5, line 48 - column 8, 
line 22 and figures 3 and 4); 

Falcon et al., US 6,632,094 Bl, (previously cited) teaches synchronized text 
highlighting with audio narration for improving reading skills (abstract and 
column 8, lines 23-26); 

Sastry et al., US 6,687,877 Bl, (previously cited) teaches content 
annotation synchronized with user actions and recorded audio (column 5, lines 
39-61); 

Blume, US 6,915,103 B2, (previously cited) teaches an audio broadcast 
system to read an audio track corresponding to the position and speed of a 
handheld stylus (column 3, line 32-56 and figures 1-5); 

Lilienthal, US 6,933,928 Bl, (previously cited) teaches a paperless book with 
visual text that advances as the audio text is finished (column 3, lines 43-50); 

Gaddy eta I., US 7,228,189 Bl, (previously cited) teaches a method for 
synchronizing a streaming music file with a locally captured audio input 
(abstract, column 7, line 39 - column 8, line 36, and figures 3a and 3b); 

Estes et al., US 4,941,829 A, teaches a method for providing a dynamic 
tutorial for a computer program (see abstract); and 

Ricard, US 5,745,738 A, teaches another method for providing simulations 
to demonstrate the use of software (see abstract). 
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Any inquiry concerning this communication or earlier communications 
from the examiner should be directed to DANIEL R. SELLERS whose telephone 
number is (571 )272-7528. The examiner can normally be reached on Monday to 
Friday, 9am to 5:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Vivian C. Chin can be reached on (571)272-7848. The fax 
phone number for the organization where this application or proceeding is 
assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR 
only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-21 7-9197 (toll-free). If 
you would like assistance from a USPTO Customer Service Representative or 
access to the automated information system, call 800-786-9199 (IN USA OR 
CANADA) or 571 -272-1 000. 



/Daniel R. Sellers/ 
Examiner, Art Unit 2614 
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